Neural Networks with Superexpressive Activations and Integer Weights

An example of an activation function $$\sigma $$ is given such that networks with activations $$\{\sigma , \lfloor \cdot \rfloor \}$$ integer weights and a fixed architecture depending only on the input dimension d approximate continuous functions $$[0,1]^d$$ . The range required for $$\varepsilon -approximation Hölder derived, which, together our discrete choice weights, allows to obtain number needed attain approximation rate. Combining this obtained speed applying oracle inequality we get prediction rate $$n^{\frac{-2\beta }{2\beta +d}}\log _2n$$ neural network regression estimation unknown $$\beta -Hölder n samples. Thus, up logarithmic factor $$\log attained coincides minimax error -smooth functions. As sizes are their integers, constructed not easily encodable but they also reduce problem finding best predictor simple procedure minimization over finite set candidates.